1. Field of the Invention
The present invention relates to a coordinate system setting jig for a visual sensor used in detecting the position and attitude of an object in controlling an automatic machine such as a robot, for example, and also to a sensor coordinate system setting method using the jig.
2. Description of the Related Art
A visual sensor, which includes a camera having a CCD element array and the like and an image processing device for storing and analyzing an image picked up by means of the camera, is often used for the control of an automatic machine, such as a robot, or other similar applications. Conventionally known is a system arranged as follows. For example, in performing welding, transportation, and other operations for the workpiece by using the robot, a deviation of a workpiece as an object of operation, from its reference position or reference attitude is detected by means of one or a plurality of visual sensors, the data indicative of such deviation is transmitted as correction data for the robot position or attitude to a robot controller, and the position or attitude is corrected in accordance with the correction data as the robot executes a taught operation.
Such a system must be arranged so that a coordinate system (camera coordinate system, more specifically CCD pixel array, for example) on which the image data for the object of operation recognized by each visual sensor is based and a coordinate system (usually work coordinate system) set in the robot can be integrated, and position data (pixel value data) detected based on the former coordinate system by the individual visual sensors can be converted into data (work coordinate system data) represented by the latter coordinate system.
Usually, therefore, a sensor coordinate system, which can be recognized in common by the visual sensors and the robot controller, is established so that the pixel value data detected by the visual sensors are temporarily converted into data on the sensor coordinate system, and the resulting data are further converted into data on the workpiece coordinate system on which actual robot drive control is based. (In some cases, the sensor coordinate system may be made to be identical with the work coordinate system.)
As can be inferred from the above example of application to the control of the robot, in constructing a system intended for utilizing for various purposes the object position information detected by the visual sensors, it is usually necessary for such detected information to be arranged for the conversion into objective data. More specifically, the information, such as the pixel value data, which can only be recognized directly by the visual sensors, need to be arranged so that they can be converted into the information or the data which can be represented by an objective coordinate system. In the description to follow, the objective coordinate system will be referred to generally as the sensor coordinate system without regard to the application.
In order to convert the pixel value data into the data represented by the predetermined sensor coordinate system, it is sufficient to have the sensor coordinate system recognized by the visual sensors in any way, utilizing the CPU, image processor, memories, etc. included in the visual sensor so that the correlation between the sensor coordinate system and the pixel value can be stored in the visual sensor. After this coordinate system is established once, the data represented on the sensor coordinate system can be obtained with respect to the object recognized by the visual sensor, in accordance with the aforesaid correspondence relationship stored in the visual sensor, unless the sensor coordinate system is changed. Even in case the sensor coordinate system is changed, the sensor coordinate system set in the visual sensor can be changed by utilizing a conversion program as long as the relationship between new and old sensor coordinate systems is known.
Conventionally, in a common method for setting such sensor coordinate system, a sensor coordinate system setting jig with three feature points thereon is located in a predetermined position (e.g., fixed position on a fixed plane in a work space), the three feature points, whose positions on the sensor coordinate system are known, are caught in the field of view of the camera of the visual sensor, the pixel value data for the individual feature points are stored; and, based on these data and the known position data, the visual sensor recognizes the origin and the directions and scales (actual size of one pixel on the plane of the object, or reciprocal of scale factor on the pixel plane) of individual coordinate axes (normally X- and Y-axes) of the sensor coordinate system.
There is another method in which the feature points are increased to four or more, e.g., seven, in number, in consideration of that object is recognized through the lens system of the camera of the visual sensor, these seven feature points are recognized by the visual sensor, and the sensor coordinate system is established in the same processings. (Tsai's camera model indicates that at least seven feature points are needed when the lens system is used.)
According to the prior art described above, all of the three or more feature points must be recognized by means of the visual sensor in catching a plurality of feature points on the jig in the field of view of the camera of the visual sensor and executing the sensor coordinate system setting operation.
If the feature points are as few as three or thereabout, it is relatively easy to catch all the feature points in the camera view field of the visual sensor and coordinate the pixel data corresponding thereto to the data of the individual feature points on the sensor coordinate system. Since the lens system has aberration, it is inevitably difficult to set the sensor coordinate system accurately in consideration of image distortion which is liable to occur in the peripheral area of the field of view, in particular.
This drawback can be avoided by increasing the feature points so that they are distributed throughout the field of view of the camera. If this is done, however, another problem such that all the feature points cannot always be contained in the field of view will be encountered. For example, if an application to the robot control is assumed, for example, the actual size of the field of view of the visual sensor will inevitably have to be changed correspondingly when the size or operation area of the object of operation or required position detecting accuracy is changed.
Thus, in the case of high-accuracy position detection for small workpieces locally confined to a narrow area, the camera is located in a relatively close position, so that the actual size of the field of view is accordingly small. Thus it is necessary to prepare a small-sized jig having feature points arranged with a high array density in a narrow area.
Consequently, in the case of relatively rough position detection for large workpieces scattered in a wide area, the camera is located at a larger distance, so that the actual size of the field of view is accordingly wide. To meet this, therefore, it is desirable to prepare a large-sized jig having feature points arranged with a low array density in a wide area.
Thus, according to the prior art system, jigs of various sizes must be prepared depending on the nature of the operation, and besides, the kinds of programs to be provided for coordinate setting increase correspondingly, and the reusability of the programs lowers.
Furthermore, coupled with the previously discussed problem of the size of the field of view, when any of the feature points cannot be recognized satisfactorily, general state of the array of the feature points cannot be covered easily. Thus, it is difficult to recognize the position of the origin and the directions of the coordinate axes, and this entails a problem such that the coordinate setting itself cannot be executed normally.